3D Transrectal Ultrasound Images Research Articles

Deep learning-based ultrasound auto-segmentation of the prostate with brachytherapy implanted needles.

Accurate segmentation of the clinical target volume (CTV) corresponding to the prostate with or without proximal seminal vesicles is required on transrectal ultrasound (TRUS) images during prostate brachytherapy procedures. Implanted needles cause artifacts that may make this task difficult and time-consuming. Thus, previous studies have focused on the simpler problem of segmentation in the absence of needles at the cost of reduced clinical utility. To use a convolutional neural network (CNN) algorithm for segmentation of the prostatic CTV in TRUS images post-needle insertion obtained from prostate brachytherapy procedures to better meet the demands of the clinical procedure. A dataset consisting of 144 3-dimensional (3D) TRUS images with implanted metal brachytherapy needles and associated manual CTV segmentations was used for training a 2-dimensional (2D) U-Net CNN using a Dice Similarity Coefficient (DSC) loss function. These were split by patient, with 119 used for training and 25 reserved for testing. The 3D TRUS training images were resliced at radial (around the axis normal to the coronal plane) and oblique angles through the center of the 3D image, as well as axial, coronal, and sagittal planes to obtain 3689 2D TRUS images and masks for training. The network generated boundary predictions on 300 2D TRUS images obtained from reslicing each of the 25 3D TRUS images used for testing into 12 radial slices (15° apart), which were then reconstructed into 3D surfaces. Performance metrics included DSC, recall, precision, unsigned and signed volume percentage differences (VPD/sVPD), mean surface distance (MSD), and Hausdorff distance (HD). In addition, we studied whether providing algorithm-predicted boundaries to the physicians and allowing modifications increased the agreement between physicians. This was performed by providing a subset of 3D TRUS images of five patients to five physicians who segmented the CTV using clinical software and repeated this at least 1week apart. The five physicians were given the algorithm boundary predictions and allowed to modify them, and the resulting inter- and intra-physician variability was evaluated. Median DSC, recall, precision, VPD, sVPD, MSD, and HD of the 3D-reconstructed algorithm segmentations were 87.2 [84.1, 88.8]%, 89.0 [86.3, 92.4]%, 86.6 [78.5, 90.8]%, 10.3 [4.5, 18.4]%, 2.0 [-4.5, 18.4]%, 1.6 [1.2, 2.0]mm, and 6.0 [5.3, 8.0]mm, respectively. Segmentation time for a set of 12 2D radial images was 2.46 [2.44, 2.48]s. With and without U-Net starting points, the intra-physician median DSCs were 97.0 [96.3, 97.8]%, and 94.4 [92.5, 95.4]% (p<0.0001), respectively, while the inter-physician median DSCs were 94.8 [93.3, 96.8]% and 90.2 [88.7, 92.1]%, respectively (p<0.0001). The median segmentation time for physicians, with and without U-Net-generated CTV boundaries, were 257.5 [211.8, 300.0]s and 288.0 [232.0, 333.5]s, respectively (p=0.1034). Our algorithm performed at a level similar to physicians in a fraction of the time. The use of algorithm-generated boundaries as a starting point and allowing modifications reduced physician variability, although it did not significantly reduce the time compared to manual segmentations.

Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound

Three-dimensional (3D) transrectal ultrasound (TRUS) is utilized in prostate cancer diagnosis and treatment, necessitating time-consuming manual prostate segmentation. We have previously developed an automatic 3D prostate segmentation algorithm involving deep learning prediction on radially sampled 2D images followed by 3D reconstruction, trained on a large, clinically diverse dataset with variable image quality. As large clinical datasets are rare, widespread adoption of automatic segmentation could be facilitated with efficient 2D-based approaches and the development of an image quality grading method. The complete training dataset of 6761 2D images, resliced from 206 3D TRUS volumes acquired using end-fire and side-fire acquisition methods, was split to train two separate networks using either end-fire or side-fire images. Split datasets were reduced to 1000, 500, 250, and 100 2D images. For deep learning prediction, modified U-Net and U-Net++ architectures were implemented and compared using an unseen test dataset of 40 3D TRUS volumes. A 3D TRUS image quality grading scale with three factors (acquisition quality, artifact severity, and boundary visibility) was developed to assess the impact on segmentation performance. For the complete training dataset, U-Net and U-Net++ networks demonstrated equivalent performance, but when trained using split end-fire/side-fire datasets, U-Net++ significantly outperformed the U-Net. Compared to the complete training datasets, U-Net++ trained using reduced-size end-fire and side-fire datasets demonstrated equivalent performance down to 500 training images. For this dataset, image quality had no impact on segmentation performance for end-fire images but did have a significant effect for side-fire images, with boundary visibility having the largest impact. Our algorithm provided fast (<1.5 s) and accurate 3D segmentations across clinically diverse images, demonstrating generalizability and efficiency when employed on smaller datasets, supporting the potential for widespread use, even when data is scarce. The development of an image quality grading scale provides a quantitative tool for assessing segmentation performance.

Deep-Learning-Based Multi-Organ Auto-Segmentation on 3D Transrectal Ultrasound for Ultrasound-Guided Prostate Brachytherapy

Ultrasound (US) imaging has been widely used in image-guided prostate brachytherapy. Current prostate brachytherapy uses transrectal US (TRUS) images for implant guidance, where contours of prostate and organs-at risk are necessary for treatment planning and dose evaluation. This work aims to develop a deep learning-based method for pelvic multi-organ TRUS segmentation to improve TRUS-guided prostate brachytherapy.We developed an anchor-free mask convolutional neural network (CNN) that consist of three subnetworks, i.e., a backbone, a fully convolutional one-state object detector (FCOS) head, and a mask head. The backbone extracts multi-level and multi-scale features from US image. The FOCS head utilizes these features to detect the location of volume-of-interest (VOI) of each organ. In contrast to previous mask regional CNN (Mask R-CNN) method, which perform detection and segmentation within several pre-defined potential VOIs (called as anchors), FCOS head is anchor-free, which can capture spatial correlation of multi-organs due to whole image input. Mask head performs segmentation on each detected VOI, where a spatial attention strategy is integrated into mask head that helps to focus on informative features that can well-represent the organ boundary and suppress noise introduced by previous subnetwork and by arrival TRUS image itself. For evaluation, we retrospectively investigated 80 prostate cancer patients by a five-fold cross-validation. Prostate, bladder, rectum and urethra were segmented and compared with manual contours using Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), and mean surface distance (MSD).For all patients, the DSC, HD95 and MSD were 0.93 ± 0.03, 2.28 ± 0.64 mm and 0.57 ± 0.20 mm for prostate; 0.75 ± 0.12, 2.58 ± 0.7 mm and 1.26 ± 0.23 mm for bladder; 0.90 ± 0.07, 1.65 ± 0.52 mm and 0.34 ± 0.16 mm for rectum; and 0.86 ± 0.07, 1.85 ± 1.71 mm and 0.44 ± 0.32mm for urethra. The proposed method outperforms two state-of-art methods (U-Net and Mask R-CNN) with better agreement with physicians' manual contours and less mis-identified speckles.We proposed an automatic deep learning-based pelvic multi-organ segmentation on 3D TRUS images for TRUS-guided prostate brachytherapy. The proposed method could provide fast and accurate multi-organ segmentations for prostate, bladder, rectum and urethra, and outperformed two competing methods. This deep-learning-based multi-organ auto-segmentation may play an important role in future clinical practice to improve auto-planning and auto-evaluation during TRUS-guided prostate brachytherapy.

Feasibility of fusing three-dimensional transabdominal and transrectal ultrasound images for comprehensive intraoperative visualization of gynecologic brachytherapy applicators.

In this study, we propose combining three-dimensional (3D) transrectal ultrasound (TRUS) and 3D transabdominal ultrasound (TAUS) images of gynecologic brachytherapy applicators to leverage the advantages of each imaging perspective, providing a broader field-of-view and allowing previously obscured features to be recovered. The aim of this study was to evaluate the feasibility of fusing these 3D ultrasound (US) perspectives based on the applicator geometry in a phantom prior to clinical implementation. In proof-of-concept experiments, 3D US images of application-specific multimodality pelvic phantoms were acquired with tandem-and-ring and tandem-and-ovoids applicators using previously validated imaging systems. Two TRUS images were acquired at different insertion depths and manually fused based on the position of the ring/ovoids to broaden the TRUS field-of-view. The phantom design allowed "abdominal thickness" to be modified to represent different body habitus and TAUS images were acquired at three thicknesses for each applicator. The merged TRUS images were then combined with TAUS images by rigidly aligning applicator components and manually refining the registration using the positions of source channels and known tandem length, as well as the ring diameter for the tandem-and-ring applicator. Combined 3D US images were manually, rigidly registered to images from a second modality (magnetic resonance (MR) imaging for the tandem-and-ring applicator and X-ray computed tomography (CT) for the tandem-and-ovoids applicator (based on applicator compatibility)) to assess alignment. Four spherical fiducials were used to calculate target registration errors (TREs), providing a metric for validating registrations, where TREs were computed using root-mean-square distances to describe the alignment of manually identified corresponding fiducials. An analysis of variance (ANOVA) was used to identify statistically significant differences (p<0.05) between the TREs for the three abdominal thicknesses for each applicator type. As an additional indicator of geometric accuracy, the bladder was segmented in the 3D US and corresponding MR/CT images, and volumetric differences and Dice similarity coefficients (DSCs) were calculated. For both applicator types, the combination of 3D TRUS with 3D TAUS images allowed image information obscured by the shadowing artifacts under single imaging perspectives to be recovered. For the tandem-and-ring applicator, the mean ± one standard deviation (SD) TREs from the images with increasing thicknesses were 1.37 ± 1.35mm, 1.84 ± 1.22mm, and 1.60 ± 1.00mm. Similarly, for the tandem-and-ovoids applicator, the mean ± SD TREs from the images with increasing thicknesses were 1.37 ± 0.35mm, 1.95 ± 0.90mm, and 1.61 ± 0.76mm. No statistically significant difference was detected in the TREs for the three thicknesses for either applicator type. The mean volume differences for the bladder segmentations were 3.14% and 2.33% and mean DSCs were 87.8% and 87.7% for the tandem-and-ring and tandem-and-ovoids applicators, respectively. In this proof-of-concept study, we demonstrated the feasibility of fusing 3D TRUS and 3D TAUS images based on the geometry of tandem-and-ring and tandem-and-ovoids applicators. This represents a step toward an accessible and low-cost 3D imaging method for gynecologic brachytherapy, with the potential to extend this approach to other intracavitary configurations and hybrid applicators.

Prostate specific membrane antigen positron emission tomography for lesion-directed high-dose-rate brachytherapy dose escalation

Prostate specific membrane antigen positron emission tomography imaging (PSMA-PET) has demonstrated potential for intra-prostatic lesion localization. We leveraged our existing database of co-registered PSMA-PET imaging with cross sectional digitized pathology to model dose coverage of histologically-defined prostate cancer when tailoring brachytherapy dose escalation based on PSMA-PET imaging. Using a previously-developed automated approach, we created segmentation volumes delineating underlying dominant intraprostatic lesions for ten men with co-registered pathology-imaging datasets. To simulate realistic high-dose-rate brachytherapy (HDR-BT) treatments, we registered the PSMA-PET-defined segmentation volumes and underlying cancer to 3D trans-rectal ultrasound images of HDR-BT cases where 15 Gray (Gy) was delivered. We applied dose/volume optimization to focally target the dominant intraprostatic lesion identified on PSMA-PET. We then compared histopathology dose for all high-grade cancer within whole-gland treatment plans versus PSMA-PET-targeted plans. Histopathology dose was analyzed for all clinically significant cancer with a Gleason score of 7or greater. The standard whole-gland plans achieved a median [interquartile range] D98 of 15.2 [13.8-16.4] Gy to the histologically-defined cancer, while the targeted plans achieved a significantly higher D98 of 16.5 [15.0-19.0] Gy (p=0.007). This study is the first to use digital histology to confirm the effectiveness of PSMA-PET HDR-BT dose escalation using automatically generated contours. Based on the findings of this study, PSMA-PET lesiondose escalation can lead to increased dose to the ground truth histologically defined cancer.

Automatic prostate segmentation using deep learning on clinically diverse 3D transrectal ultrasound images.

Needle-based procedures for diagnosing and treating prostate cancer, such as biopsy and brachytherapy, have incorporated three-dimensional (3D) transrectal ultrasound (TRUS) imaging to improve needle guidance. Using these images effectively typically requires the physician to manually segment the prostate to define the margins used for accurate registration, targeting, and other guidance techniques. However, manual prostate segmentation is a time-consuming and difficult intraoperative process, often occurring while the patient is under sedation (biopsy) or anesthetic (brachytherapy). Minimizing procedure time with a 3D TRUS prostate segmentation method could provide physicians with a quick and accurate prostate segmentation, and allow for an efficient workflow with improved patient throughput to enable faster patient access to care. The purpose of this study was to develop a supervised deep learning-based method to segment the prostate in 3D TRUS images from different facilities, generated using multiple acquisition methods and commercial ultrasound machine models to create a generalizable algorithm for needle-based prostate cancer procedures. Our proposed method for 3D segmentation involved prediction on two-dimensional (2D) slices sampled radially around the approximate central axis of the prostate, followed by reconstruction into a 3D surface. A 2D U-Net was modified, trained, and validated using images from 84 end-fire and 122 side-fire 3D TRUS images acquired during clinical biopsies and brachytherapy procedures. Modifications to the expansion section of the standard U-Net included the addition of 50% dropouts and the use of transpose convolutions instead of standard upsampling followed by convolution to reduce overfitting and improve performance, respectively. Manual contours provided the annotations needed for the training, validation, and testing datasets, with the testing dataset consisting of 20 end-fire and 20 side-fire unseen 3D TRUS images. Since predicting with 2D images has the potential to lose spatial and structural information, comparisons to 3D reconstruction and optimized 3D networks including 3D V-Net, Dense V-Net, and High-resolution 3D-Net were performed following an investigation into different loss functions. An extended selection of absolute and signed error metrics were computed, including pixel map comparisons [dice similarity coefficient (DSC), recall, and precision], volume percent differences (VPD), mean surface distance (MSD), and Hausdorff distance (HD), to assess 3D segmentation accuracy. Overall, our proposed reconstructed modified U-Net performed with a median [first quartile, third quartile] absolute DSC, recall, precision, VPD, MSD, and HD of 94.1 [92.6, 94.9]%, 96.0 [93.1, 98.5]%, 93.2 [88.8, 95.4]%, 5.78 [2.49, 11.50]%, 0.89 [0.73, 1.09]mm, and 2.89 [2.37, 4.35]mm, respectively. When compared to the best-performing optimized 3D network (i.e., 3D V-Net with a Dice plus cross-entropy loss function), our proposed method performed with a significant improvement across nearly all metrics. A computation time <0.7s per prostate was observed, which is a sufficiently short segmentation time for intraoperative implementation. Our proposed algorithm was able to provide a fast and accurate 3D segmentation across variable 3D TRUS prostate images, enabling a generalizable intraoperative solution for needle-based prostate cancer procedures. This method has the potential to decrease procedure times, supporting the increasing interest in needle-based 3D TRUS approaches.

Multi-needle Localization with Attention U-Net in US-guided HDR Prostate Brachytherapy.

Ultrasound (US)-guided high dose rate (HDR) prostate brachytherapy requests the clinicians to place HDR needles (catheters) into the prostate gland under transrectal US (TRUS) guidance in the operating room. The quality of the subsequent radiation treatment plan is largely dictated by the needle placements, which varies upon the experience level of the clinicians and the procedure protocols. Real-time plan dose distribution, if available, could be a vital tool to provide more subjective assessment of the needle placements, hence potentially improving the radiation plan quality and the treatment outcome. However, due to low signal-to-noise ratio (SNR) in US imaging, real-time multi-needle segmentation in 3D TRUS, which is the major obstacle for real-time dose mapping, has not been realized to date. In this study, we propose a deep learning-based method that enables accurate and real-time digitization of the multiple needles in the 3D TRUS images of HDR prostate brachytherapy. A deep learning model based on the U-Net architecture was developed to segment multiple needles in the 3D TRUS images. Attention gates were considered in our model to improve the prediction on the small needle points. Furthermore, the spatial continuity of needles was encoded into our model with total variation (TV) regularization. The combined network was trained on 3D TRUS patches with the deep supervision strategy, where the binary needle annotation images were provided as ground truth. The trained network was then used to localize and segment the HDR needles for a new patient's TRUS images. We evaluated our proposed method based on the needle shaft and tip errors against manually defined ground truth and compared our method with other state-of-art methods (U-Net and deeply supervised attention U-Net). Our method detected 96% needles of 339 needles from 23 HDR prostate brachytherapy patients with 0.290±0.236mm at shaft error and 0.442±0.831mm at tip error. For shaft localization, our method resulted in 96% localizations with less than 0.8mm error (needle diameter is 1.67mm), while for tip localization, our method resulted in 75% needles with 0mm error and 21% needles with 2mm error (TRUS image slice thickness is 2mm). No significant difference is observed (P=0.83) on tip localization between our results with the ground truth. Compared with U-Net and deeply supervised attention U-Net, the proposed method delivers a significant improvement on both shaft error and tip error (P<0.05). We proposed a new segmentation method to precisely localize the tips and shafts of multiple needles in 3D TRUS images of HDR prostate brachytherapy. The 3D rendering of the needles could help clinicians to evaluate the needle placements. It paves the way for the development of real-time plan dose assessment tools that can further elevate the quality and outcome of HDR prostate brachytherapy.

A partial augmented reality system with live ultrasound and registered preoperative MRI for guiding robot-assisted radical prostatectomy.

We propose an image guidance system for robot assisted laparoscopic radical prostatectomy (RALRP). A virtual 3D reconstruction of the surgery scene is displayed underneath the endoscope's feed on the surgeon's console. This scene consists of an annotated preoperative Magnetic Resonance Image (MRI) registered to intraoperative 3D Trans-rectal Ultrasound (TRUS) as well as real-time sagittal 2D TRUS images of the prostate, 3D models of the prostate, the surgical instrument and the TRUS transducer. We display these components with accurate real-time coordinates with respect to the robot system. Since the scene is rendered from the viewpoint of the endoscope, given correct parameters of the camera, an augmented scene can be overlaid on the video output. The surgeon can rotate the ultrasound transducer and determine the position of the projected axial plane in the MRI using one of the registered da Vinci instruments. This system was tested in the laboratory on custom-made agar prostate phantoms. We achieved an average total registration accuracy of 3.2 ± 1.3mm. We also report on the successful application of this system in the operating room in 12 patients. The average registration error between the TRUS and the da Vinci system for the last 8 patients was 1.4 ± 0.3mm and average target registration error of 2.1 ± 0.8mm, resulting in an in vivo overall robot system to MRI mean registration error of 3.5mm or less, which is consistent with our laboratory studies.

Weakly-supervised convolutional neural networks for multimodal image registration.

One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels.

Detection of Prostate Cancer Using Radial/Axial Scanning of 2D Trans-rectal Ultrasound Images

The search for improvement in the result of segmentation of regions of interest in medical images has continued to be a source of challenge to researchers. Several research efforts have gone in to delineate regions of interest in the prostate gland from Trans-rectal ultrasound (TRUS) 2D-images. In this work, we develop a fast algorithm based on radial/axial scanning of the pixels of the prostate gland image with the goal of detecting hyper-echoic pixels that are bound within the boundaries of the gland TRUS 2D-images. The algorithm implements expert knowledge and utilizes the features extracted from the intensity of the TRUS images, primarily the relative intensity and gradient to delineate region of interest. It employs radial/axial scanning of the image from common seed point automatically selected to detect the region of the gland and subsequently hyper-echoic pixels which indicate suspected cancerous tissue cites. Evaluation of the algorithm performance was done by comparing detection result with that of expert radiologists. The detection algorithm gave an average accuracy of 88.55% and sensitivity of 71.65%.

Accuracy and variability of high-dose-rate prostate brachytherapy needle tip localization using live two-dimensional and sagittally reconstructed three-dimensional ultrasound

PurposeTo measure the accuracy and variability of manual high-dose-rate (HDR) prostate brachytherapy (BT) needle tip localization using sagittally reconstructed three-dimensional (3D) transrectal ultrasound (TRUS) augmented with live two-dimensional (2D) sagittal TRUS. Methods and MaterialsTen prostate cancer patients underwent HDR-BT during which the sagittally assisted sagittally reconstructed (SASR) segmentation technique was completed in parallel with commercially available sagittally assisted axially reconstructed (SAAR) TRUS for comparison. The SASR technique makes use of live 2D ultrasound intraoperatively and allows needle tip updates using the final 3D image in the absence of image artifacts. These updates were repeated offline twice by two separate users. Needle end-length measurements were used to calculate insertion depth errors (IDEs) for each technique. ResultsImages of 147 needles were analyzed. For the SASR technique, both users were confident in tip positions on the final 3D image within 3 mm for 52% of needles, so these tip positions were updated. For the remaining 48% of needles, the tip positions from the live 2D images were used. This SASR technique enabled the localization of all needles with IDEs within ±3 mm for 84% of needles and IDE range of [−6.2 mm, 5.9 mm], compared with 57% and [−8.1 mm, 7.7 mm] when using the commercially available SAAR technique. ConclusionsThe SASR technique mitigates the impact of 3D TRUS image artifacts on HDR-BT needle tip localization by incorporating live 2D sagittal TRUS intraoperatively and provides a statistically significant reduction in IDE variance compared with the routine SAAR technique.

A random walk-based segmentation framework for 3D ultrasound images of the prostate.

Accurate segmentation of the prostate on ultrasound images has many applications in prostate cancer diagnosis and therapy. Transrectal ultrasound (TRUS) has been routinely used to guide prostate biopsy. This manuscript proposes a semiautomatic segmentation method for the prostate on three-dimensional (3D) TRUS images. The proposed segmentation method uses a context-classification-based random walk algorithm. Because context information reflects patient-specific characteristics and prostate changes in the adjacent slices, and classification information reflects population-based prior knowledge, we combine the context and classification information at the same time in order to define the applicable population and patient-specific knowledge so as to more accurately determine the seed points for the random walk algorithm. The method is initialized with the user drawing the prostate and non-prostate circles on the mid-gland slice and then automatically segments the prostate on other slices. To achieve reliable classification, we use a new adaptive k-means algorithm to cluster the training data and train multiple decision-tree classifiers. According to the patient-specific characteristics, the most suitable classifier is selected and combined with the context information in order to locate the seed points. By providing accuracy locations of the seed points, the random walk algorithm improves segmentation performance. We evaluate the proposed segmentation approach on a set of 3D TRUS volumes of prostate patients. The experimental results show that our method achieved a Dice similarity coefficient of 91.0%±1.6% as compared to manual segmentation by clinically experienced radiologist. The random walk-based segmentation framework, which combines patient-specific characteristics and population information, is effective for segmenting the prostate on ultrasound images. The segmentation method can have various applications in ultrasound-guided prostate procedures.

Toward a 3D transrectal ultrasound system for verification of needle placement during high-dose-rate interstitial gynecologic brachytherapy.

Treatment for gynecologic cancers, such as cervical, recurrent endometrial, and vaginal malignancies, commonly includes external-beam radiation and brachytherapy. In high-dose-rate (HDR) interstitial gynecologic brachytherapy, radiation treatment is delivered via hollow needles that are typically inserted through a template on the perineum with a cylinder placed in the vagina for stability. Despite the need for precise needle placement to minimize complications and provide optimal treatment, there is no standard intra-operative image-guidance for this procedure. While some image-guidance techniques have been proposed, including magnetic resonance (MR) imaging, X-ray computed tomography (CT), and two-dimensional (2D) transrectal ultrasound (TRUS), these techniques have not been widely adopted. In order to provide intra-operative needle visualization and localization during interstitial brachytherapy, we have developed a three-dimensional (3D) TRUS system. This study describes the 3D TRUS system and reports on the system validation and results from a proof-of-concept patient study. To obtain a 3D TRUS image, the system rotates a conventional 2D endocavity transducer through 170 degrees in 12s, reconstructing the 2D frames into a 3D image in real-time. The geometry of the reconstruction was validated using two geometric phantoms to ensure the accuracy of the linear measurements in each of the image coordinate directions and the volumetric accuracy of the system. An agar phantom including vaginal and rectal canals, as well as a model uterus and tumor, was designed and used to test the visualization and localization of the interstitial needles under idealized conditions by comparing the needles' positions between the 3D TRUS scan and a registered MR image. Five patients undergoing HDR interstitial gynecologic brachytherapy were imaged using the 3D TRUS system following the insertion of all needles. This image was manually, rigidly registered to the clinical postinsertion CT scan based on the vaginal cylinder of the needle template. The positions of the tips and the trajectory of the needle paths were compared between the modalities. The observed geometric errors of the system were ≤0.3mm in each of the three coordinate planes of the 3D US image and the mean measured volumetric error was 0.10cm3 . In the phantom study, the mean needle tip difference was 1.54±0.71 mm and the mean trajectory difference was 0.94±0.89 degrees (n=14). In the invivo study, a total of 73 needles were placed, of which 88% of needles were visible and 79% of tips were identifiable in the 3D TRUS images. Six of the nine needles that were not visible were due to shadowing artifacts created by the presence of the vaginal cylinder of the needle template. The mean distance between corresponding needle tips in the two modalities was 3.82±1.86mm and the mean trajectory difference was 3.04±1.63 degrees for the five patients. In this proof-of-concept study, the 3D TRUS system allowed for localization of needles not obscured by shadowing artifacts, providing a method for visualizing needles intra-operatively during HDR interstitial brachytherapy of gynecologic cancers and providing the potential for 3D image-guidance.

Personalized Biomechanical Modeling of Prostate Deformation Based on Elastography For Mr-Trus Image Registration

To evaluate the clinical significance of personalized biomechanical modeling of prostate deformation based on ultrasound elastography for magnetic resonance imaging (MRI)-Transrectal Ultrasound (TRUS) image registration. A total number of 35 patients and 5 commercial prostate phantom were imaged via transrectal ultrasound elastography, TRUS and MRI at the first affiliated hospital of SYSU from June 2014 to December 2015. We propose a personalized biomechanical model via the patient-specific ultrasound elastography for the deformable registration of prostate MR and 3D TRUS images. The registration accuracy was evaluated by the target registration error (TRE) and also the t-test was conducted to validate the statistical significance of our results. All the 35 sets of patient data, as well as the phantom data, were successfully registered. The TRE value of the phantom data was 1.69mm. The mean TRE value of 35 patients was 1.29 mm, compared with the 2.56 mm TRE value of the registration method without patient-specific biomechanical properties via elastography, was approximately 48% lower. The improvement was with statistical significance validated by t-test (p-value was 1.45e-8). Personalized biomechanical modeling of prostate deformation based on ultrasound elastography for MR- TRUS image registration is with clinical significance and is a promising way to provide more quality guidance and improve the accuracy of prostate biopsy.

Random Walk Based Segmentation for the Prostate on 3D Transrectal Ultrasound Images.

This paper proposes a new semi-automatic segmentation method for the prostate on 3D transrectal ultrasound images (TRUS) by combining the region and classification information. We use a random walk algorithm to express the region information efficiently and flexibly because it can avoid segmentation leakage and shrinking bias. We further use the decision tree as the classifier to distinguish the prostate from the non-prostate tissue because of its fast speed and superior performance, especially for a binary classification problem. Our segmentation algorithm is initialized with the user roughly marking the prostate and non-prostate points on the mid-gland slice which are fitted into an ellipse for obtaining more points. Based on these fitted seed points, we run the random walk algorithm to segment the prostate on the mid-gland slice. The segmented contour and the information from the decision tree classification are combined to determine the initial seed points for the other slices. The random walk algorithm is then used to segment the prostate on the adjacent slice. We propagate the process until all slices are segmented. The segmentation method was tested in 32 3D transrectal ultrasound images. Manual segmentation by a radiologist serves as the gold standard for the validation. The experimental results show that the proposed method achieved a Dice similarity coefficient of 91.37±0.05%. The segmentation method can be applied to 3D ultrasound-guided prostate biopsy and other applications.

Transrectal ultrasound for image-guided adaptive brachytherapy in cervix cancer – An alternative to MRI for target definition?

PurposeTo compare the maximum high risk clinical target volume (CTVHR) dimensions and image quality between magnetic resonance imaging (MRI), transrectal ultrasound (TRUS) and computed tomography (CT) in image guided adaptive brachytherapy (IGABT) of locally advanced cervical cancer. Material and methodsAll patients with locally advanced cervical cancer treated with radiochemotherapy and IGABT between 09/2012-05/2013 were included in this study. T2-weighted MRI (1.5 tesla), TRUS and CT were performed before (MRIpreBT, TRUSpreBT) and/or after (MRIBT, TRUSBT and CTBT) insertion of the applicator. 3D TRUS image acquisition was done with a customized US stepper device and software. The HR CTV was defined on 3D image sequences acquired with different imaging modalities by one blinded observer, in accordance to the GEC-ESTRO recommendations for MRI-based target volume delineation, as the complete cervical mass including the tumour, any suspicious areas of parametrial involvement and the normal cervical stroma. Maximum HR CTV width and thickness were measured on transversal planes. Image quality was classified using the following scoring system: Grade 0: not depicted, Grade 1: inability to discriminate, margin not recognizable, Grade 2: fair discrimination, margin indistinct, Grade 3: excellent discrimination, margin distinct. Descriptive statistics, mean differences between the groups, with MRIBT as reference, and a paired t-test were calculated. ResultsImages from 19 patients (FIGO IB: 3, IIB: 9, IIIB: 5, IVB: 2) were available for analysis. The mean difference in maximum HR CTV width of TRUSBT, TRUSpreBT, MRIpreBT, CTBT to MRIBT was 0.0mm±4.7 (n.s.), −1.1mm±5.6 (n.s.), 0.7mm±6.4 (n.s.) and 13.8mm±6.7 (p<0.001). The mean difference in maximum HR CTV thickness of TRUSBT, TRUSpreBT, MRIpreBT, CTBT to MRIBT was -3.4mm±5.9 (p=0.037), −3.4mm±4.2 (p<0.001), 2.0mm±6.1 (n.s.) and 13.9mm±6.3 (p<0.001). Mean scores of image quality of the target volume was 2.9 for TRUSpreBT, 2.3 for TRUSBT, 2.9 for MRIpreBT, 2.7 for MRIBT and 2.1 for CTBT. ConclusionFor the assessment of the HR CTV in IGABT of cervical cancer, TRUS is within the intraobserver variability of MRI. TRUS is superior to CT as it yields systematically smaller deviations from MRI, with good to excellent image quality. Small differences of TRUS HR CTV thickness are likely related to differences in image slice orientation and compression of the cervix by the TRUS probe before insertion of the brachytherapy applicator.

Rotationally resliced 3D prostate TRUS segmentation using convex optimization with shape priors.

Efficient and accurate segmentations of 3D end-firing transrectal ultrasound (TRUS) images play an important role in planning of 3D TRUS guided prostate biopsy. However, poor image quality of the input 3D TRUS images, such as strong imaging artifacts and speckles, often makes it a challenging task to extract the prostate boundaries accurately and efficiently. In this paper, the authors propose a novel convex optimization-based approach to delineate the prostate surface from a given 3D TRUS image, which reduces the original 3D segmentation problem to a sequence of simple 2D segmentation subproblems over the rotational reslices of the 3D TRUS volume. Essentially, the authors introduce a novel convex relaxation-based contour evolution approach to each 2D slicewise image segmentation with the joint optimization of shape information, where the learned 2D nonlinear statistical shape prior is incorporated to segment the initial slice, its result is propagated as a shape constraint to the segmentation of the following slices. In practice, the proposed segmentation algorithm is implemented on a GPU to achieve the high computational performance. Experimental results using 30 patient 3D TRUS images show that the proposed method can achieve a mean Dice similarity coefficient of 93.4% ± 2.2% in 20 s for one 3D image, outperforming the existing local-optimization-based methods, e.g., level-set and active-contour, in terms of accuracy and efficiency. In addition, inter- and intraobserver variability experiments show its good reproducibility. A semiautomatic segmentation approach is proposed and evaluated to extract the prostate boundary from 3D TRUS images acquired by a 3D end-firing TRUS guided prostate biopsy system. Experimental results suggest that it may be suitable for the clinical use involving the image guided prostate biopsy procedures.

Prostate CT segmentation method based on nonrigid registration in ultrasound-guided CT-based HDR prostate brachytherapy.

The technological advances in real-time ultrasound image guidance for high-dose-rate (HDR) prostate brachytherapy have placed this treatment modality at the forefront of innovation in cancer radiotherapy. Prostate HDR treatment often involves placing the HDR catheters (needles) into the prostate gland under the transrectal ultrasound (TRUS) guidance, then generating a radiation treatment plan based on CT prostate images, and subsequently delivering high dose of radiation through these catheters. The main challenge for this HDR procedure is to accurately segment the prostate volume in the CT images for the radiation treatment planning. In this study, the authors propose a novel approach that integrates the prostate volume from 3D TRUS images into the treatment planning CT images to provide an accurate prostate delineation for prostate HDR treatment. The authors' approach requires acquisition of 3D TRUS prostate images in the operating room right after the HDR catheters are inserted, which takes 1-3 min. These TRUS images are used to create prostate contours. The HDR catheters are reconstructed from the intraoperative TRUS and postoperative CT images, and subsequently used as landmarks for the TRUS-CT image fusion. After TRUS-CT fusion, the TRUS-based prostate volume is deformed to the CT images for treatment planning. This method was first validated with a prostate-phantom study. In addition, a pilot study of ten patients undergoing HDR prostate brachytherapy was conducted to test its clinical feasibility. The accuracy of their approach was assessed through the locations of three implanted fiducial (gold) markers, as well as T2-weighted MR prostate images of patients. For the phantom study, the target registration error (TRE) of gold-markers was 0.41 ± 0.11 mm. For the ten patients, the TRE of gold markers was 1.18 ± 0.26 mm; the prostate volume difference between the authors' approach and the MRI-based volume was 7.28% ± 0.86%, and the prostate volume Dice overlap coefficient was 91.89% ± 1.19%. The authors have developed a novel approach to improve prostate contour utilizing intraoperative TRUS-based prostate volume in the CT-based prostate HDR treatment planning, demonstrated its clinical feasibility, and validated its accuracy with MRIs. The proposed segmentation method would improve prostate delineations, enable accurate dose planning and treatment delivery, and potentially enhance the treatment outcome of prostate HDR brachytherapy.

Interactive Prostate Shape Reconstruction from 3D TRUS Images.

This paper presents a two-step, semi-automated method for reconstructing a three-dimensional (3D) shape of the prostate from a 3D transrectal ultrasound (TRUS) image. While the method has been developed for prostate ultrasound imaging, it can potentially be applicable to any other organ of the body and other imaging modalities. The proposed method takes as input a 3D TRUS image and generates a watertight 3D surface model of the prostate. In the first step, the system lets the user visualize and navigate through the input volumetric image by displaying cross sectional views oriented in arbitrary directions. The user then draws partial/full contours on selected cross sectional views. In the second step, the method automatically generates a watertight 3D surface of the prostate by fitting a deformable spherical template to the set of user-specified contours. Since the method allows the user to select the best cross-sectional directions and draw only clearly recognizable partial or full contours, the user can avoid time-consuming and inaccurate guesswork on where prostate contours are located. By avoiding the usage of noisy, incomprehensible portions of the TRUS image, the proposed method yields more accurate prostate shapes than conventional methods that demand complete cross-sectional contours selected manually, or automatically using an image processing tool. Our experiments confirmed that a 3D watertight surface of the prostate can be generated within five minutes even from a volumetric image with a high level of speckles and shadow noises.

Magnetic resonance imaging-targeted, 3D transrectal ultrasound-guided fusion biopsy for prostate cancer: Quantifying the impact of needle delivery error on diagnosis.

Magnetic resonance imaging (MRI)-targeted, 3D transrectal ultrasound (TRUS)-guided "fusion" prostate biopsy intends to reduce the ∼23% false negative rate of clinical two-dimensional TRUS-guided sextant biopsy. Although it has been reported to double the positive yield, MRI-targeted biopsies continue to yield false negatives. Therefore, the authors propose to investigate how biopsy system needle delivery error affects the probability of sampling each tumor, by accounting for uncertainties due to guidance system error, image registration error, and irregular tumor shapes. T2-weighted, dynamic contrast-enhanced T1-weighted, and diffusion-weighted prostate MRI and 3D TRUS images were obtained from 49 patients. A radiologist and radiology resident contoured 81 suspicious regions, yielding 3D tumor surfaces that were registered to the 3D TRUS images using an iterative closest point prostate surface-based method to yield 3D binary images of the suspicious regions in the TRUS context. The probabilityP of obtaining a sample of tumor tissue in one biopsy core was calculated by integrating a 3D Gaussian distribution over each suspicious region domain. Next, the authors performed an exhaustive search to determine the maximum root mean squared error (RMSE, in mm) of a biopsy system that gives P ≥ 95% for each tumor sample, and then repeated this procedure for equal-volume spheres corresponding to each tumor sample. Finally, the authors investigated the effect of probe-axis-direction error on measured tumor burden by studying the relationship between the error and estimated percentage of core involvement. Given a 3.5 mm RMSE for contemporary fusion biopsy systems,P ≥ 95% for 21 out of 81 tumors. The authors determined that for a biopsy system with 3.5 mm RMSE, one cannot expect to sample tumors of approximately 1 cm(3) or smaller with 95% probability with only one biopsy core. The predicted maximum RMSE giving P ≥ 95% for each tumor was consistently greater when using spherical tumor shapes as opposed to no shape assumption. However, an assumption of spherical tumor shape for RMSE = 3.5 mm led to a mean overestimation of tumor sampling probabilities of 3%, implying that assuming spherical tumor shape may be reasonable for many prostate tumors. The authors also determined that a biopsy system would need to have a RMS needle delivery error of no more than 1.6 mm in order to sample 95% of tumors with one core. The authors' experiments also indicated that the effect of axial-direction error on the measured tumor burden was mitigated by the 18 mm core length at 3.5 mm RMSE. For biopsy systems with RMSE ≥ 3.5 mm, more than one biopsy core must be taken from the majority of tumors to achieveP ≥ 95%. These observations support the authors' perspective that some tumors of clinically significant sizes may require more than one biopsy attempt in order to be sampled during the first biopsy session. This motivates the authors' ongoing development of an approach to optimize biopsy plans with the aim of achieving a desired probability of obtaining a sample from each tumor, while minimizing the number of biopsies. Optimized planning of within-tumor targets for MRI-3D TRUS fusion biopsy could support earlier diagnosis of prostate cancer while it remains localized to the gland and curable.